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RELATED APPLICATIONS 

This application relates to, incorporates by reference, and claims priority from. 
United States Patent Application ##/###,###, entitled, "Method and Apparatus for Content 
Personalization Over a Telephone Interface," filed October 22, 1999, having inventors 
5 Hadi Partovi, Roderick S. Brathwaite, Angus M, Davis, Micahel S. McCue, Brandon W. 
Porter, John Giannandrea, Eckart Walther, Anthony Accardi, and Zhe Li.. 

BACKGROUND OF THE INVENTION 

Field of the Invention 

This invention relates to the field of streaming content. In particular, the invention 
10 relates to technologies for providing streaming content to users over a telephone interface. 
The invention also relates to identifying and registering users using telephone identifying 
information and personalizing the content, including the streaming content, presented to 
them using a profile selected using the telephone identifying information. 

1 5 Description of the Related Art 

The following describes streaming content available over computer networks. 
Then various telephone systems are described. 

Streaming content is multimedia data sent as a stream of data. Typically, the 
content is broken into small parts, compressed, sent over a network, uncompressed, and 
20 then played on a user's computer. This means that users can begin viewing, or listening to, 
the content before the entire stream is received. Examples of streaming media systems 
include streaming audio and video systems from Real Networks, Inc., and Microsoft 
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Corporation, and Apple Computer, Inc. One problem with such systems is that the 
streaming data can only be accessed by users' computers. 

The following describes various techniques used in telephone systems to provide 
enhanced user features. First, telephone identifying information will be discussed. Many 
telephone systems that support enhanced user features use telephone identifying 
information as a basic component. Then, a variety of example systems will be discussed 
that use telephone identifying information to provide enhanced user features will be 
discussed, 

1 . Telephone Identifying Information 

The advent of automatic number identification (ANI) and calling number 
identification (CNID, CLID, or CID) within the North American Numbering Plan (NANP) 
has supported the creation of a number of services that use these pieces of telephone 
identifying information. Comparable systems may be used in other numbering plans and 
countries to support similar services. 

For example, when consumers receive credit cards in the mail, they have to call 
from their home telephone numbers to activate the cards. This is a typical use of ANI. In 
this instance, the credit card company matches the ANI information provided when the 
consumer calls to a previously provided telephone number. If the ANI matches the credit 
card company's records, the credit card company activates the card. 
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2. Examples of Telephone System Personalization 

a. Personalization Generally 

With the adyent of widely available real-time delivery of telephone identifying 
information such as ANI, a number of systems have been developed to use that 
5 information. One of the most common uses of ANI is for credit card activation. However, 
previous systems have been single purpose and typically require reference to other 
information provided separately. For example, credit card activation lines require 
separately provided information, e.g. your home phone number from the application. 

b. Building Personalized Content on the Web 

1 0 Some systems allow a user to build personalized content over the web. One 

example is the my Yahoo!™ service provided by Yahoo! of Santa Clara, California at 
<http://my.yahoo.com/>. The personalized content pages developed on the web are 
delivered over the web to users accessing the pages with computers. These systems rely on 
a usemame and password type system to identify the user rather than telephone identifying 

1 5 information and the delivery mechanisms is different. 

c. Interactive Personalization 

Still other systems allow users to personalize the content without entering special 
editing modes. For example, Amazon.com, of Seattle, Washington, keeps track of your 
purchases and preferences using cookies stored on a customer's web browser. 
20 Some telephone systems provide limited customization capabilities. For example, 

voice mail systems from Octel, a division of Lucent Technologies, allow a user to set 
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preferences for prompt length, but those settings must be made explicitly by each user. 
Further, customization is limited to a few options like prompt length and outgoing 
message selection. The user can not redefine the way the voice mail system works for 
her/him beyond those narrow customization options. Further, these customizations do not 
5 affect the kinds of content and further the presentation is not selected based on telephone 
identifying information. 

d. Locale Selection 

Services such as Amtrak's 1-800-USA-RAIL reservation line use telephone 
identifying information to select an initial region. For example, if you call Amtrak's 
1 0 reservation number in the Northeastem United States, the system presents options relating 
to the Boston- Washington line. However, if you call from California, the system presents 
information about travel between San Francisco and Los Angeles. 

This can be accomplished by using the calling party's area code and/or exchange 
included with the telephone identifying information to select a region. The area codes 
15 and/or exchanges can then be paired to different scripts or default selections. For example, 
the area codes for New York City, e.g. "212", could be mapped to the Northeast Corridor 
while San Francisco, "415", could be mapped to the San Francisco-Los Angeles line. 
However this does not change the kind of content presented and it is not user-selected. 

e. Time Appropriate Information Presentation 

20 Several services provide information through the telephone. That information may 

be adapted based on the time of day or date. 
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Some systems provide the information irrespective of the telephone identifying 
information. One example is Moviefone™, 777-FILM in most locales. Moviefone™ uses 
the current time at the called number to present appropriate information. The called 
number can be obtained using the dialed number identification service (DNIS). Thus, if 
5 you call Moviefone™ in the San Francisco Bay Area at 10 o'clock in the morning, only 
movies starting after 10 o'clock in the morning in the San Francisco Bay Area will be 
presented to you. However, if you call the Philadelphia Moviefone™, +1 (215) 222-FILM, 
from California, you will hear the Philadelphia movie times in Eastern Time. Thus, at 
10 o'clock in the morning Pacific Time, a call to the Philadelphia Moviefone™ will 
10 produce information for Philadelphia show times after one o'clock in the afternoon Eastern 
Time at Philadelphia area theatres. 

f. Targeted Advertising 

Some firee long distance services provide customized advertising to support their 
services. One example is FreeWay™ offered by Broadpoint, of Landover, Maryland, 

1 5 <http://www.broadpoint.coni/>. These services require an explicit user registration 
process, typically using a computer to access a web site, to provide the service with a 
profile. Once the profile is provided, the advertising is targeted to the particular person's 
explicitly provided demographic information. In some instances, the advertising may be 
targeted both based on the caller's demographics and their location. Thus, callers fi-om the 

20 San Francisco Bay Area with a particular explicit demographic profile may be presented 
one ad, while callers from outside the San Francisco Bay Area may be presented with 
another ad. Another, similar, service is offered on by phone by UAccess, Inc., 
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<http://www.uaccess.com/>, by calling +1 (800) UACCESS, and provides consumers 
targeted advertising based on profile information they enter. 

Voice Character 

Most telephone systems have a small number of voice actors. Continuing with the 
5 example of Moviefone™, one actor performs all of the menus and prompts. Other systems 
may use different voice actors for different subsystems. 

These actors are typically selected on a system wide basis and as such, different 
voices, talents, speeds, characteristics, dialects, and other prosody aspects of the 
presentation are not user selectable. 

10 h. Purchase Recommendations 

Voice systems such as GALAXY from the Massachusetts Institute of Technology 
(MIT), Cambridge, Massachusetts, have been adapted to provide information about 
purchasing decisions for used cars. For example, GALAXY has been used to allow for 
interactive browsing of automobile classified ads. These voice systems are problem 
1 5 domain specific. Further, the systems are designed to locate vehicles matching a particular 
set of criterion, rather than making actual recommendations. 

Other systems are web based. For example, Amazon.com will make book 
suggestions for users connected to the web via a computer. However, those suggestions 
are limited to a particular site, e.g. Amazon.com. 
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i. Voice Login 

Most telephone systems require a user to explicitly identify herself/himself by 
using a combination of a login identifier, e.g. credit card number, account number, etc., 
and a personal identification number (PEST). Some systems abbreviate this process by 
5 allowing a user calling from a particular phone to shortcut this process slightly. For 

example, callers using a phone number associated with a particular credit card might only 
be asked to enter the last four digits of their credit card number together with their billing 
zip code instead of all sixteen digits of the card number. Other products such as Nuance 
Verfier™ from Nuance Communications, Menlo Park, California, support voice login 
10 capabilities, e.g. you just speak instead of entering a password. 

i. Initial Profile Generation from Database Lookups 

Most systems that provide information over the telephone require users to 
explicitly answer one or more questions in one form or another, e.g. over the phone, the 
web, and/or in written form. These questions form a demographic and/or psychographic 
15 profile for the user. All of these systems require the user to explicitly provide her/his 
profile information. 

SUMMARY OF THE INVENTION 

A method and apparatus for providing streaming content over telephones is 
described. The creation of a voice portal is supported by the invention. Embodiments of 
20 the invention allows users to place a telephone call to access the voice portal. The user can 
access many different types of content. This content can include text based content which 
is read to the user by a text to speech system (e.g., news reports, stock prices, text content 
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of Internet sites), audio content which can be played to the user (e.g., voicemail messages, 
music), and streaming audio content (e.g., Internet broadcast radio shows, streaming news 
reports, and streaming Uve broadcasts). This content can be accessed from many different 
places. For example, the content can be retrieved from a news feed, a local streaming 
content server, an audio repository, and/or an Intemet based streaming content server. 

The streaming content allows the user to access live web broadcasts even though 
the user may not have access to a computer. The system allows audio signals from 
multiple sources to be combined to enhance the user experience. Additionally, the user can 
control many aspects of the delivery of the streaming content. 

In one embodiment, a computer system supports the voice portal functionality. The 
system can include an interface to the Public Switched Telephone Network and a system 
for interfacing with the Intemet. The PSTN interface supports communications with the 
users' telephones, while the Internet interface provides for connectivity with the Intemet. 
Between the two interfaces are a number of subsystems that support the selection, 
conversion, mixing and/or control of audio content (and in particular, streaming audio 
content). Importantly, these subsystems also support the personalization of the content. 
The foUov^ng describes some of the personalization features of various embodiments of 
the invention. 

The personalized content presented during a telephone call is specific to that user 
based on the profile associated with her/his telephone identifying information. For 
example, if a user, John, has previously called from the telephone number 650-493-####, 
he may have indicated he prefers a Southem dialect. Then, upon subsequent calls to the 
system from his telephone John will be greeted in a Southem dialect based on the profile 
associated with his nxmiber. 
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Example personalizations provided by embodiments of the invention will now be 
described. The system may personalize the session based on the time and/or date as 
determined from the telephone identifying information. For example, based on the local 
time for the calling party, time and/or date appropriate options may be presented. For 
5 example, if a user calls from California at noon, a restaurant for lunch may be suggested. 
However, a caller from London at that same moment might be presented evening 
entertairmient selections because of the eight hour time difference. 

The system may personalize the session based on the caller's locale as determined 
from the telephone identifying information. For example, caller's from Palo Alto, 

10 California may hear different selections and options than callers from Washington, D.C. 
This may include locale specific events, e.g. a county fair, locale specific annovmcements, 
e.g. flood watch for Santa Clara Coimty, etc. 

The system may target advertising based on the caller's demographic and/or 
psychographic profile. Additionally, the advertising may be targeted based on the 

15 telephone identifying information. For example, overall demographic information for a 
particular area code-exchange combination may be used, e.g. 650-493 corresponds to Palo 
Alto, California, with an average income of $X. On an international scale, this can be used 
to the extent that the particular numbering plan can be combined with relevant 
geographic/demographic/psychographic information about callers. Both types of targeted 

20 advertising allow the callers to be qualified, e.g. match the requirements of the advertiser. 
For example, a San Francisco jewelry store might only want to reach households in the 
Bay Area with an average household income exceeding $50,000 a year. The targeted 
advertising can ensure that callers presented with the ad are qualified. 
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The system may adapt the voice character, e,g, the speech patterns and dialect, of 
the system according to the caller's telephone identifying information and/or the caller's 
ovm voice character. Thus, for example, for callers who speak more slowly the system 
may reduce the speed at which the system speaks and/or increase the volume. This may be 
5 based on the telephone identifying information, e.g. hospital, an explicit request by the 
user, e.g. "Slow dovm", and/or implicitly based on the caller's speech pattems and 
interactions with the system. 

Based on the caller's profile ~ as retrieved through the telephone identifying 
information ~ and/or demographic information from other sources, e.g. locale based 
10 and/or reverse phone directory lookup, the system can make purchasing suggestions. For 
example, close to a holiday like Mother's day, the system may suggest a gift based on what 
other people in that locale, e.g. Palo Alto, are buying and or based on the user's own 
purchasing history, e.g. she/he bought flowers last year. Similarly, the voice portal can 
recommend the purchase of an audio CD based on previous audio CD purchases. 
1 5 The system may use a voice password and/or touch-tone login system when 

appropriate to distinguish the caller or verify the caller's identity for specific activities. 

These customizations may be combined in a variety of fashions. Thus, for 
example, the time, the locale, and a preferred dialect may be used to present a purchase 
recommendation. 

20 Profiles can be constructed implicitly as the caller uses embodiments of the 

invention as well as through explicit designation of preferences. For example, the user 
might specify an existing personalized web site to use in building her/his profile for the 
voice system. Similarly, for a caller from New York who repeatedly asks for the weather 



C VNRPORTBLVPALibl\krr\1098505.2 



13 



Attorney Docket Number 22379-702 



in San Francisco, the system might automatically include the San Francisco weather in the 
standard weather report without explicit specification, or confirmation. 

Additionally, new callers may have an initial profile generated based on one or 
more database lookups for demographic information based on their telephone identifying 
information. 
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BRIEF DESCRIPTION OF THE FIGURES 

Fig. 1 illustrates a system including embodiments of the invention used to provide 
streaming contents from the Internet to a telephone. 

Fig. 2 illustrates the components of a voice portal supporting streaming content 
5 delivery and personalized content. 

Fig. 3 is a process flow diagram supporting personalization and registration of and 
for users accessing a voice portal over a telephone interface. 

Fig. 4 is a process flow diagram for personalizing a voice portal over a web based 
interface. 

10 Fig. 5 is a process flow diagram for providing personalized content according to 

some embodiments of the invention. 

Fig. 6 is a process flow diagram for providing streaming content to a telephone. 
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DETAILED DESCRIPTION 

A, Introduction 

A voice portal for presenting streaming content over a telephone interface is 
described. The voice portal allows users of telephones, including cellular telephones, to 
5 access a voice portal by dialing a phone number to listen to streaming content. The 
information provided over the voice portal may come from the World Wide Web 
(WWW), databases, third parties, and/or other sources. 

The term voice portal refers to the capability of various embodiments of the 
invention to provide customized voice and/or audio content services to a caller. The voice 
10 portal can receive dual-tone multi-frequency (DTMF or touch-tone) commands as well as 
spoken commands to further control the voice and/or audio content presented and the 
manner of presentation. Note that not all embodiments of the invention use a voice portal, 
and in particular, not all embodiments use a voice portal with personalization capabilities. 
Generally, any system that provides a telephone interface, an Internet interface, and can 
1 5 send an extracted audio signal, from streaming content being received by the Intemet 
interface, to the telephone interface, would support the invention. 

Embodiments of the invention use telephone identifying information to personalize 
caller interactions with the voice portal. This allows the system to present highly 
customized information to each caller based on a personal profile the system associates 
20 with the telephone identifying information. 

The invention will be described in greater detail as follows. First, a number of 
definitions useful to understanding the invention are presented. Then, the hardware and 
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software architecture is presented in the System Overview. Then, a series of sections 
describe the various personalization features provided by different embodiments of the 
invention. Then, a section describes the streaming content capabilities of the system. 

B. Definitions 

5 1 . Telephone Identifying Information 

For the purposes of this application, the term telephone identifying information 
will be used to refer to ANI information, CID information, and/or some other technique 
for automatically identifying the source of a call and/or other call setup information. For 
example, ANI information typically includes a dialed number identification service 

10 (DNIS). Similarly, CID information may include text data including the subscriber's name 
and/or address, e.g. "Jane Doe". Other examples of telephone identifying information 
might include the type of calling phone, e.g. cellular, pay phone, and/or hospital phone. 

Additionally, the telephone identifying information may include wireless carrier 
specific identifying information, e.g. location of wireless phone now, etc. Also, signaling 

1 5 system seven (SS7) information may be included in the telephone identifying information, 

2. User Profile 

A user profile is a collection of information about a particular user. The user 
profile typically includes collections of different information as shown and described more 
fully in connection v^th Figure 6. Notably, the user profile contains a combination of 
20 explicitly made selections and implicitly made selections. 
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Explicitly made selections in the user profile stem from requests by the user to the 
system. For example, the user might add business news to the main topic list. Typically, 
explicit selections come in the form of a voice, or touch-tone command, to save a 
particular location, e.g. "Remember this", "Bookmark it", "shortcut this", pound (#) key 
5 touch-tone, etc., or through adjustments to the user profile made through the web interface 
using a computer. 

In contrast, implicit selections come about through the conduct and behavior of the 
user. For example, if the user repeatedly asks for the weather in Palo Alto, California, the 
system may automatically provide the Palo Alto weather report without further prompting. 

10 In other embodiments, the user may be prompted to confirm the system's implicit choice, 
e.g. the system might prompt the user "Would you like me to include Palo Alto in the 
standard weather report from now on?" 

Additionally, the system may allow the user to customize the system to meet 
her/his needs better. For example, the user may be allowed to control the verbosity of 

15 prompts, the dialect used, and/or other settings for the system. These customizations can 
be made either explicitly or implicitly. For example if the user is providing commands 
before most prompts are finished, the system could recognize that a less verbose set of 
prompts is needed and implicitly set the user's prompting preference to briefer prompts. 

3. Topics and Content 

20 A topic is any collection of similar content. Topics may be arranged hierarchically 

as well. For example, a topic might be business news, while subtopics might include stock 
quotes, market report, and analyst reports. Within a topic different types of content are 
available. For example, in the stock quotes subtopic, the content might include stock 
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quotes. The distinction between topics and the content within the topics is primarily one of 
degree in that each topic, or subtopic, will usually contain several pieces of content. 

4. Qualified 

The term qualified as it is used in this application refers to whether or not a 
particular user being presented an advertisement, or other material, meets the demographic 
and/or psychographic profile requirements for that advertisement, or content. For example, 
a San Francisco-based bookstore might request that all listeners to its advertisement be 
located in a particular part of the San Francisco Bay Area. Thus, a user of the system 
would be qualified if she lived in the designated part of the San Francisco Bay Area, 

Different embodiments of the invention may qualify users of the system according 
to different requirements. For example, in some instances advertising, or content, is 
qualified solely based on telephone identifying information. In other embodiments the 
telephone identifying information is used in conjunction with other information such as an 
associated user profile, a reverse telephone number lookup for locale demographics, 
and/or other information. 

5. Locale 

As used in this application, the term locale refers to any geographic area. The 
geographic area may be a neighborhood, a city, a county, a metropolitan region, a state, a 
country, a continent, a group of countries, and/or some other collection of one or more 
geographic areas, e.g. all United State major metropolitan areas. 

For this reason, a single user of the system may be considered to be in several 
locales. For example, a caller from Palo Alto, California, might be in the Palo Alto locale, 
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a Silicon Valley locale, a San Francisco Bay Area locale, a Northern California locale, a 
California state locale, and a United States locale. 

Thus, the telephone identifying information for a single telephone number can be 
mapped to a number of system-defined locales. 

5 6. Voice Character 

The term voice character as it is used in this application refers to all aspects of 
speech pronunciation including dialect, speed, volume, gender of speaker, pitch, language, 
voice talent used, actor, characteristics of speech, and/or other prosody values. Users can 
adjust the voice character of the system by changing their voice character settings. 
10 For example, an elderly user could select voice character settings that provide 

louder volume and slower speech. Similarly, a caller from the South could adjust the voice 
character settings to support a Southern dialect. 

7. Demographic and Psvchographic Profiles 

Both demographic profiles and psychographic profiles contain information relating 
15 to a user. Demographic profiles typically include factual information, e.g. age, gender, 
marital status, income, etc. Psychographic profiles typically include information about 
behaviors, e.g. fun loving, analytical, compassionate, fast reader, slow reader, etc. As used 
in this application, the term demographic profile will be used to refer to both demographic 
and psychographic profiles. 
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8. Streaming Content 

Streaming technology allows a user to see, and/or listen to, streaming content as it 
is being downloaded. Streaming content is audio and/or video data sent as a stream. 
Typically, the content is broken into small parts, compressed, sent over a network, 
5 uncompressed, and then played on a user's computer. This means that users can begin 
viewing, or listening to, the content before the entire stream is received. Note, video data 
can include animations. Importantly, streaming data, as opposed to non-streaming data, 
does not require the user to download an entire data file before the user can begin listening 
to or viewing the data. 

10 

C. Svstem Overview 

First, the hardware and software architecture of a system including an embodiment 
of the invention will be described with reference to Figures 1-2. Figure 1 illustrates a 
system including embodiments of the invention used to provide streaming content and 

1 5 personalized content to users of telephones according to telephone identifying information. 
The system of Figure 1 can be used to allow users of standard telephones and cellular 
telephones to access a voice portal with streaming content from their telephones. 

The following lists the elements of Figure 1 and describes their interconnections. 
Figure 1 includes a telephone 100, a cellular telephone 101, a computer 102, a telephone 

20 network 104, an Internet 106, a telephone gateway 107, a web server 108, a voice portal 
110, a shared database 1 12, a personalized site 130, a streaming content server 150, a local 
streaming content server 170 and a streaming content path 170. The cellular telephone 101 
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and the telephone 100 are coupled in communication with the telephone network 104. The 
telephone network 104 is coupled in communication with the telephone gateway 107. The 
telephone gateway 107 is coupled in communication with the voice portal 110. The 
computer 102 and the streaming content server 150 is coupled in communication with the 
Internet 106. The Internet 106 is coupled in communication with the web server 108. The 
voice portal 1 10 and the web server 108 are coupled in communication with the shared 
database 1 12. The personalized site 130 is coupled in communication with the Intemet 
106. The local streaming content server 160 is coupled in communication with the voice 
portal 110. 

The foUov^ng describes each of the elements of Figure 1 in greater detail. The use 
of each of the elements will be described further in conjunction with the sections 
describing the personalization features. 

The telephone 100 and the cellular telephone 101 are two different telephone 
interfaces to the voice portal 110. The telephone 100 and the cellular telephone 101 may 
be any sort of telephone and/or cellular telephone. For example the telephone 100 or the 
cellular telephone 101 may be a land line phone, a PBX telephone, a satellite phone, a 
wireless telephone, and/or any other type of communication device capable of providing 
voice commxmication and/or touch-tone signals over the telephone network 104. However, 
any audio signal carrying interface could be used. 

The telephone network 104 may be the public switched telephone network (PSTN) 
and/or some other type of telephone network. For example, some embodiments of the 
invention may allow users with a voice over Intemet Protocol (IP) phone to access the 
voice portal 110. The telephone network 104 is coupled to the telephone gateway 107 that 
allows the voice communications and/or touch-tone signals from the telephone network 
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104 to reach the voice poital 1 10 in usable form. Similarly, the telephone gateway 107 
allows audio signals generated by the voice portal 1 10 to be sent over the telephone 
network 104 to respective telephones, e.g. the telephone 100. The telephone network 104 
generally represents an audio signal carrying network. 

The computer 102 is a computer such as a personal computer, a thin client 
computer, a server computer, a handheld computer, a set top box computer, and/or some 
other type of visual web browsing device. The computer 102 is coupled in coramunication 
with the Intemet 106, e.g. by a dial-up connection, a digital subscriber loop (DSL), a cable 
modem, and/or some other type of connection. This allows the computer 102 to 
communicate with the web server 108. The computer 102 typically provides a visual 
interface to the WWW and the web server 108 using web browsing software such as 
Intemet Explorer™ from Microsoft Corporation, Redmond, Washington. 

Both the web server 108 and the voice portal 1 10 are capable of communicating 
with the shared database 1 12 to register users, build personal profiles implicitly and/or 
explicitly as will be described more fiiUy below. The database 1 12 stores profiles for each 
user based on an association between one or more pieces of telephone identifying 
information and a particular user. Thus, the database may have a profile for a user Sarah 
Smith that is keyed to her home telephone number, e.g. 650-493-####. Additionally, Sarah 
could associate other numbers, e.g. work, cellular, etc., with her profile either implicitly, 
e.g. by repeatedly calling the voice portal 1 10 from those numbers, or explicitly, e.g. by 
adding those numbers to the system directly. 

In some embodiments, an existing profile for a web-based portal is adapted for use 
by the voice portal 1 10 by associating one or more telephone numbers with the existing 
profile as stored in the shared database 1 12. In these embodiments, the existing profile 
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may be further modified for use with the voice portal 1 10 to allow for different 
preferences between the web and the voice interfaces. 

The call flow arrows 114-122 shown on Figure 1 will be described in greater detail 

below. 

5 The streaming content server 170 represents any computer system capable of 

providing streaming content. The streaming content server 170 could be a server provided 
by Real Networks, Microsoft Corporation, Apple Computer, Inc., or any number of 
streaming content deliver systems. The local content server 160 is similar to the streaming 
content server 170 except that the local content server 160 is directly accessible to the 

1 0 voice portal 1 10 (i.e., not requiring an Internet access). 

The streaming content path 170 represents logical paths by which streaming 
content can be delivered telephones. The streaming content path 170 shows that the 
streaming content is accessed from the streaming content server 150, or the local 
streaming content server 160, sent through the voice portal 1 10, the telephone gateway 

15 107 and the telephone network 104, to be delivered to the telephone 100 (or cellular phone 
101). 

Figure 2 illustrates the components of a voice portal supporting streaming and 
personalized content. This could be used to support the voice portal 110. 

The following lists the elements of Figure 2 and describes their intercormections. 
20 The voice portal 1 10 is coupled in communication with the telephone gateway 107. The 
voice portal 1 1 0 includes a call manager 200, an execution engine 202, a data connectivity 
engine 220, an evaluation engine 222 and a streaming engine 224. Additionally Figure 2 
includes elements that may be included in the voice portal 1 10, or which may be separate 
from, but coupled to, the voice portal 110. Thus Figure 2 also includes a recognition server 
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210, a text to speech server 214, an audio repository 212, the local streaming content 
server 160, the shared database 1 12, a database 226, the Internet 106, a database 228 and a 
web site 230. The call manager 200 within the voice portal 1 1 0 is coupled to the execution 
engine 202. The execution engine 202 is coupled to the recognition server 210, the text to 
speech server 214, the audio repository 212, data connectivity engine 220, the evaluation 
engine 222 and the streaming engine 224. The voice portal 1 10 is coupled in 
communication with the shared database 1 12, the database 226 and the Internet 106. The 
Internet 106 is coupled in commimication with the streaming content server 150 and the 
database 228 and the web site 230. 

The following describes each of the elements of Figure 2 in greater detail. The use 
of each of the elements will be described further in conjunction with the sections 
describing the personalization features and the streaming content features. 

Typically, the voice portal 1 10 is implemented using one or more computers. The 
computers may be server computers such as UNIX workstations, personal computers 
and/or some other type of computers. Each of the components of the voice portal 1 10 may 
be implemented on a single computer, multiple computers and/or in a distributed fashion. 
Thus, each of the components of the voice portal 1 10 is a functional unit that may be 
divided over multiple computers and/or multiple processors. The voice portal 110 
represents an example of a telephone interface subsystem. Different components may be 
included in a telephone interface subsystem. For example, a telephone interface subsystem 
may include one or more of the following components: the call manager 200, the 
execution engine, the data connectivity 220, the evaluation engine 222, the streaming 
engine 224, the audio repository 212, the text to speech 214 and/or the recognition engine 
210. 
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The call manager 200 is responsible for scheduling call and process flow among 
the various components of the voice portal 110. The call manager 200 sequences access to 
the execution engine 202. Similarly, the execution engine 202 handles access to the 
recognition server 210, the text to speech server 214, the audio repository 212, the data 
connectivity engine 220, the evaluation engine 222 and the streaming engine 224. 

The recognition server 210 supports voice, or speech, recognition. The recognition 
server 210 may use Nuance 6™ recognition software from Nuance Communications, 
Menlo Park, California, and/or some other speech recognition product. The execution 
engine 202 provides necessary grammars to the recognition server 210 to assist in the 
recognition process. The results from the recognition server 210 can then be used by the 
execution engine 202 to further direct the call session. Additionally, the recognition server 
1 10 may support voice login using products such as Nuance Verifier™ and/or other voice 
login and verification products. 

The text to speech server 214 supports the conversion of text to synthesized speech 
for transmission over the telephone gateway 107. For example, the execution engine 202 
could request that the phrase, "The temperature in Palo Alto, California, is currently 58 
degrees and rising" be spoken to a caller. That phrase would be translated to speech by the 
text to speech server 214 for playback over the telephone network on the telephone (e.g. 
the telephone 100). Additionally the text to speech server 214 may respond using a 
selected dialect and/or other voice character settings appropriate for the caller. 

The audio repository 212 may include recorded sounds and/or voices. In some 
embodiments the audio repository 212 is coupled to one of the databases (e.g. the database 
226, the database 228 and/or the shared database 1 12) for storage of audio files. Typically, 
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the audio repository server 212 responds to requests from the execution engine 202 to play 
a specific sound or recording. 

For example, the audio repository 212 may contain a standard voice greeting for 
callers to the voice portal 110, in which case the execution engine 202 could request 
play-back of that particular sound file. The selected sound file would then be delivered by 
the audio repository 212 through the call manager 200 and across the telephone gateway 
107 to the caller on the telephone, e.g. the telephone 100. Additionally, the telephone 
gateway 107 may include digital signal processors (DSPs) that support the generation of 
sounds and/or audio mixing. Some embodiments of the invention include telephony 
systems from Dialogics, an Intel Corporation. 

The execution engine 202 supports the execution of multiple threads with each 
thread operating one or more applications for a particular call to the voice portal 1 10. 
Thus, for example, if the user has called in to the voice portal 1 10, a thread may be started 
to provide her/him a voice interface to the system and for accessing other options. 

In some embodiments of the invention an extensible markup language (XML)- 
style language is used to program applications. Each application is then written in the 
XML-style language and executed in a thread on the execution engine 202. In some 
embodiments, an XML-style language such as VoiceXML from the VoiceXML Forum, 
<http://www.voicexml.org/>, is extended for use by the execution engine 202 in the voice 
portal 110. 

Additionally, the execution engine 202 may access the data connectivity engine 
220 for access to databases and web sites (e.g. the shared database 1 12, the web site 230), 
the evaluation engine 222 for computing tasks and the streaming engine 224 for 
presentation of streaming media and audio. 
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The streaming engine 224 may allow users of the voice portal 110 to access 
streaming audio content, or the audio portion of streaming video content, over the 
telephone interface. For example, a streaming media broadcast from ZDNet™ could be 
accessed by the streaming engine 224 for playback through the voice portal. The 
streaming engine 224 can act as a streaming content client to a streaming content server, 
e.g., the streaming engine 224 can act like a RealPlayer software client to receive 
streaming content broadcasts from a Real Networks server. Additionally, the streaming 
engine 224 can participate in a streaming content broadcast by acting like a streaming 
broadcast forwarding server. This second ftmction is particularly usefiil where multiple 
users are listening to the same broadcast at the same time (e.g., multiple users may call 
into the voice portal 1 10 to listen to the same live streaming broadcast of a company's 
conference call with the analysts). 

The data connectivity engine 220 supports access to a variety of databases 
including databases accessed across the Internet 106, e.g. the database 228, and also access 
to web sites over the Internet such as the web site 230. In some embodiments the data 
connectivity engine can access standard query language (SQL) databases, open database 
connectivity databases (ODBC), and/or other types of databases. The shared database 1 12 
is represented separately from the other databases in Figure 2; however, the shared 
database 1 12 may in fact be part of one of the other databases, e.g. the database 226. Thus, 
the shared database 1 12 is distinguished from other databases accessed by the voice portal 
1 10 in that it contains user profile information. 

Having described the hardware and software architecture supporting various 
embodiments of the invention, the various personalization features provided by different 
embodiments of the invention will now be described. After the personalization features. 
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the streaming content aspects of various embodiments of the invention are described in 
detail. 

D. Telephone Driven Profile Building 

Turning to Figure 3, the process of creating a profile using a telephone interface 
will be described. This process will be described with reference to the call flow arrows 
shown on Figure 1 as well. 

The voice portal 1 10 is able to flexibly handle multiple callers from a single 
telephone, e.g. Tom and Dick are roommates and both call from 650-493 -####. Similarly, 
the voice portal 1 10 is able to handle a single caller that uses multiple telephones, e.g. 
Tom has a cell phone 650-245-####, his home phone 650-493-####, and a work phone 
408-301-####. The manner in which the voice portal 1 10 can handle some of the above 
situations will be discussed throughout. In the example used while describing Figure 3, the 
process will be described using a caller Jane Smith as an exemplary caller who has never 
registered with the voice portal 110 from any telephone and an exemplary caller John Doe 
who has previously called the voice portal 110 from his telephone 100. 

First, at step 300, telephone identifying information is received. This is shown in 
Figure 1 by call flow arrow 1 14 representing the transfer of telephone identifying 
information through the telephone gateway 107 to the voice portal 110. This step occurs 
after a user has placed a call to the voice portal 110 with a telephone, e.g. the telephone 
100. 

Next, at step 302, a determination is made as to whether the telephone identifying 
information corresponds to a known profile, e.g. is the user registered? Some examples 
may be illustrative. If Jane Smith uses the cellular telephone 101 to call the voice portal 
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1 10 for the first time, her telephone identifying information will not be associated with any 
existing unique profile in the shared database 1 12. Therefore, at step 302, the 
determination would be made that she is not registered and the process would continue at 
step 304. In contrast, John Doe has previously called the voice portal ftom the telephone 
5 1 00 and so his telephone identifying information will be associated with a profile in the 
shared database 112 and the process would continue at step 306. 

If, the telephone identifying information is not associated with an existing profile 
in the shared database 1 12, a new profile is created at step 304. The new profile may be 
initialized using a variety of information derived from the telephone identifying 
10 information and/or predetermined values for the voice portal 110. Thus, for example, 
when Jane Smith calls for the first time ftom the cellular telephone 101, an initial profile 
can be created using the calling niimber, e.g. 650-493-####, included in the telephone 
identifying information to select initial profile settings. The call flow arrow 1 16 shows this 
process on Figure 1 . The use of the telephone identifying information to create an initial 
15 profile is discussed below in the section "Automatic Profile Initialization". 

In some embodiments, the profile is not initialized using the telephone identifying 
information. In other embodiments, the user may be explicitly queried by the voice portal 
1 10 to create one or more components of the initial profile, e.g. "Please speak your first 
name", to allow for more personalized prompting by the voice portal 110. Once a profile is 
20 created, the process continues at step 306. 

At step 306, the profile is retrieved ftom the shared database 1 12 as shown by the 
call flow arrow 118. The profile can be updated throughout the call based on the user's 
behavior and actions ~ implicit preferences ~ as well as explicit requests from the user to 
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customize the voice portal 110. Once a profile is selected at step 306, the personalized 
content can be presented to the user as shown by the call flow arrow 122 in Figure 1 . 

For example, John Doe, who is calling fi*om the telephone 100, already has a 
profile in the shared database 112. That profile may indicate that John prefers a southern 
5 dialect and likes to hear a quick stock market report immediately on call in. Thus, for 
John, his telephone identifying information serves to log hhn directly into the system and 
trigger the personalized behavior unique to him: a quick stock market report in a southem 
dialect. In contrast, a different caller, Sarah Brown, from a different telephone will be 
provided different personalized content based on that telephone identifying information. 

1 0 The voice portal may support multiple callers fi*om a single telephone. For 

example, Sarah Brown and John Doe may both use the telephone 100 to call the voice 
portal 1 10. In the case where two or more profiles are identified with the same telephone 
identifying information, the voice portal may prompt for a password or other unique 
identifier, either as voice or touch-tone, to select among the profiles. 

15 However, as a general matter, the voice portal is configured to minimize the need 

for a caller to provide a password. Thus, during a single call session, the caller is typically 
only asked to provide her/his password a single time. However, some embodiments of the 
invention may require that a password always be used to complete commercial 
transactions and/or after the passage of a predetermined period, e.g. ten minutes since last 

20 password prompt. In some embodiments, the user may adjust her/his profile to allow login 
without a password for playback features. 

Also, a single profile can be associated with multiple calling numbers. For 
example, the user Jane Doe could specify that both the telephone 100 and the cellular 
telephone 101 should be associated with her profile. Similarly, if Jane calls firom a new 
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telephone, e.g. pay phone, she can provide her existing telephone number and her 
password to access her profile. In some embodiments, whenever the user calls fi*om a new 
telephone number, she/he is prompted as to whether to remember the number for future 
use. In some embodiments, additional telephone identifying information, e.g. this is a pay 
5 phone, is used so that the caller is not prompted to associated telephone numbers that are 
likely to be single time uses with her/his profile. Similarly, voice verification may be used 
to recognize a caller's voice instead of, or in addition to, using a password or other 
identification number. 

Typical events that would require a password, or that the user be authenticated 
10 previously with a password, might include adding and removing items firom the user 
profile through explicit commands as well as requests for specific personal information, 
e.g. that user's stock portfolio, bank account balances, etc. 

It is not necessary for callers to the voice portal 1 10 to explicitly specify their 
preferences using this embodiment of the invention. The callers' behaviors and actions are 
15 used by the voice portal 1 10 to adopt implicit preferences, sometimes after receiving 

confirmation. For example, behaviors and actions reflecting repeated access to a content in 
a particular topic, or a particular topic, may cause the voice portal 1 10 to automatically 
include the repeatedly requested content in the default message. 

For example, if a caller from New York City repeatedly asks for the weather in San 
20 Francisco, the system can add the San Francisco weather to the standard weather report. 
Alternatively, the system may request confirmation before adding the weather report, e.g. 
"Would you Uke me to include San Francisco in the standard weather report?" Similarly, 
at the level of topics, users who repeatedly ask for information about business related 
issues may find that the system will adjust the main menu to include business. Similarly, if 
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that same user never asks for sports scores, that option may drop off the main menu. In 
some embodiments, the system may ask for confirmation before modifying the menu 
choices, or the system may notify the user of a modification and/or allow a user to 
review/change past modifications. As a result, the structure and content of the call may 
5 change, e.g. San Francisco weather will be annoixnced at the beginning of future calls and 
sports information may be omitted. 

Through the use of this process, the need for a specialized editing mode of the type 
seen on customizable web portals is reduced. The user's actions and behaviors shape the 
options presented to her/him. Thus, reducing the need to explicitly pick topics and/or 
10 content in an editing mode. However, some embodiments of the invention may allow for 
explicit profile creation in an editing mode over the web, see below, and/or over the 
telephone. Also, users are typically permitted to add and remove topics and/or items at 
will with explicit commands, e.g. "Remember this", "Remove", "Add to my stock list", 
etc. 

15 E. Web Driven Profile Building 

Turning to Figure 4, the process of modifying a profile for use over a telephone 
interface over the web will be described. This process will be described with reference to 
the call flow arrows shown on Figure 1 as well. 

The process shown in Figure 4 assximes that a profile has already been created, e.g. 
20 by calling for the first time as described above. However, in some embodiments of the 
invention, users may create profiles using the web interface by providing the telephone 
identifying information for their primary calling phone number and a password. As is the 
case with the telephone registration process described in step 304, the telephone 
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identifying information provided, here the primary calling phone number, can be used to 
create the initial profile. 

Starting at step 400, the profile is accessed using a computer (e.g. the computer 
102) via a web interface. The web interface is provided by a web server (e.g. the web 
5 server 108) and allows for access to the shared database 1 12 as shown by the call flow 
arrow 120. 

Once the user has signed in to access her/his profile on the computer, she/he can 
manually identify content and topics to build her/his profile at step 404. This can be 
supported by allowing the user to specify topics from a list of topics and then specifying 

10 per topic content from a list of content. For example, the topics might include business, 
sports, news, entertainment, and weather, to name a few. The user could include weather, 
news, and business in her/his main menu and then further customize the specific content to 
be presented within those topics. For example, within weather, the user might select the 
specific cities she/he wants listed in her/his weather menu and/or the cities for which the 

1 5 weather is automatically played. 

Alternatively, at step 402, the user can identify a web location with personalized 
content to use in building her/his profile, e.g. a uniform resource indicator (URI). For 
example. Figure 1 includes the personalized site 130. The personalized site 130 could be a 
customized portal web page, e.g. myYahoo!, My Netscape, etc., a home page the user 

20 herself/himself has designed, and/or any other web page that includes content of interest to 
the user. The user can identify the personalized site with a uniform resource indicator 
(URI), including a login identifier and password if necessary, e.g. for myYahoo! The 
personalized site 130 can then be accessed and the pertinent user preferences, e.g. news, 
stocks, selected. Taking the example of a customized portal site, the main topics selected. 
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e.g. horoscopes, and the content within, Sagittarius, could be adopted. However, the voice 
portal 110 may present its o^mi content for that particular item, e.g. the version of the 
Sagittarius horoscope on the voice portal 110 not the version from the personalized site 
130. 

5 The processes of step 402 and step 404 can be used together allowing a user to 

quickly transfer preferences from a web portal to her/his voice portal while still supporting 
explicit personalization. 

Alternatively, in some embodiments of the invention, ah existing web portal profile 
is voice enabled for use by a voice portal through the association of telephone identifying 
10 information with the existing web portal. In this embodiment, at step 402, the telephone 
identifying information, e.g. the primary calling number, is associated with an existing 
web profile, e.g. my Yahoo! profile, stored in the shared database 1 12 and that existing 
web profile is then usable from the voice portal 110 either using voice or touch-tone 
commands, 

15 Additionally, web sites like the personalized site 130 may be accessed using the 

voice portal 1 10 in some embodiments of the invention through the use of the data 
connectivity engine 220 as shown in Figure 2. 

F. Profile Building Via Other Web Sites 

Some embodiments of the invention may allow users of the voice portal 1 10 to add 
20 to their profile from other web sites. For example, if a user of the computer 102 is 

accessing a web site (e.g. the personalized site 130), the web site might include a link like 
"Add this to your voice portal." Thus, for example, from a service such as MapQuesf^^ or 
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Ameritrade™, the user could click on a link to add a particular piece of content or a 
particular topic to their portal for the voice portal 110. 

For example, a user could add her/his "QQQ" stock symbol to her/his profile on 
the voice portal 110 even though the voice portal 110 may be operated independently of 
the particular v^eb site. 

This type of web based profile building allows for widespread profile building for 
the voice portal 110 from a variety of Internet sites. Also, in some embodiments, the web 
browser software on the user's computer (e.g. the computer 102) can support an option to 
add a bookmark to the user's profile stored in the shared database 112 for the voice portal 
110. For example, a menu option in the browser on the computer 102 might include "Add 
Page to Voice Portal Shortcuts" and upon selecting that menu option, the current web page 
would be added to the user's profile on the voice portal 110. 

This would typically be accomplished by accessing a URI on the web server 108 
that included the information to be added. At that point, the web server 108 might ask for a 
primary calling phone number and/or a password. In some embodiments, a cookie stored 
by the browser on the computer 102 may be used to obviate one or both of these steps. 
After the user provides the information, or it is accepted automatically, a confirmation 
page may be shown including a retum link to the originating web page. Several example 
URI's for adding content are shown below: 

<http://www.voiceportal.com/add.cgi?topic=stock%20quote&content=QQQ> 

<h1tp://www.voiceportal.com/add.cgi?shortcut===MapQuest&ref^www.mapquest. 

<http://www.voiceportel.corn/add.cgi?shortcut==myYahoo&ref^ 

These examples are illustrative of the various types of URI's that can be placed as links on 
web sites to allow users of the voice portal 1 10 to further customize their profile. 
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G. Locale Based Personalization 

Turning to Figure 5, the basic personalization framework used by several 
embodiments of the invention is presented. At step 500, a request is made for content, or a 
topic. Then one or more of steps 502-510 take place, in parallel or sequence, and then the 
5 content is presented at step 512. Which of steps 502-5 1 0 occur for a given request may be 
determined based on the topic or content requested. For example, step 504 can be omitted 
when non-time dependent information is presented. 

Turning to step 502, the customization of content based on the calling locale. The 
telephone identifying information includes information about the caller's locale 

10 independent of any user provided registration information. This information can be 
derived from telephone routing tables that provide a descriptive name for each area 
code/exchange combination within the North American Numbering Plan (NANP). Thus, 
the phone number 650-493-#### would be associated with "Palo Alto, California". 
Similarly, 650-592-#### would be associated with "San Carlos, California". 

1 5 This information may be directly present in the telephone identifying information 

provided to the voice portal 1 10, or may be ascertained from a local exchange routing 
guide (LERG). For intemational callers outside the NANP, similar types of telephone 
identifying information can be mapped to locales within countries to the extent permitted 
by the particular numbering plan. 

20 The city-state combination may correspond to multiple locales for the purposes of 

the voice portal 110. For example, a county-wide or multi-city locale can be defined that 
encompasses multiple area code/exchange combinations. Thus, a single caller may be in 
multiple locales. 
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Locale information can be further refined through the use of additional databases, 
e.g. city/state to zip code databases, street address to five digit zip code databases, reverse 
lookup databases that map phone numbers to street addresses, longitude-latitude 
conversion databases, and/or other databases that provide locale related information from 
telephone identifying information. Thus, for example, V and H coordinates might be 
determined using the telephone identifying information. Those can be further converted to 
a longitude and latitude to determine the locale. Alternatively, a reverse phone nxmiber 
database could be used to find a specific street address for the telephone identifying 
information. 

Examples of the uses for the locale information include; providing 
locale-appropriate lottery results, providing driving directions to a requested destination, 
providing locale-appropriate weather reports, providmg locale-appropriate show times for 
movies other events, e.g. cultural, governmental, etc., traffic reports, yellow page listings, 
and/or providing other locale-related information. 

H. Time/Date Based Personalization 

Turning to step 504, the customization of content based on the time and/or date 
will now be described. The telephone identifying information includes information about 
the caller's locale independent of any user provided registration information. This 
information can be derived from telephone routing tables that provide a descriptive name 
for each area code/exchange combination within the NANP. Thus, the phone number 650- 
493-#### would be associated with "Palo Alto, California" and thus the correct time zone, 
Pacific, could be selected as well. 
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This time zone may be directly present in the telephone identifying information 
provided to the voice portal 1 10, or may be ascertained from the LERG. For international 
callers outside the NANP, similar types of telephone identifying information can be 
mapped to locales within countries to the extent permitted by the particular numbering 
5 plan. Thus, callers from United Kingdom numbers would be mapped to British Standard 
Time. 

The time zone information allows the voice portal 1 10 to customize the 
presentation of information based on the time in the caller's locale. Callers can use a single 
nationwide, or international, number to reach the voice portal 110, e.g. 800-###~####, The 
10 voice portal 110 will use the time zone information to adjust the content presented to each 
user. 

Thus, during the limch hour, the voice portal 110 might report a stock quote to the 
user while on a Friday evening, the voice portal 110 might suggest a movie. For example, 
"It is Friday night, would you be interested in seeing a movie?" A "yes" response by the 

1 5 caller will lead to the presentation of a list that is both time and date adapted and locale 
appropriate. For example, a caller from Palo Alto at six o'clock p.m. on a Friday would 
hear about show times after six o'clock p.m. in his local area. 

If necessary, the voice portal 110 may connect the user to an appropriate 
transaction system to complete a user requested transaction such a the purchase of an 

20 airline ticket, a movie ticket, an audio CD, etc. However, in many instances, the voice 
portal 110 may be able to directly complete the transaction using the data connectivity 
engine 220 and access to the Internet 106 and/or one more databases (e.g. the database 
226). This process can occur even if the caller has not explicitly provided the voice portal 
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1 10 her/his home location or the current time. For example, this personaUzed content 
might be presented immediately at after step 304 of Figure 3 in step 306. 

Similarly, other time sensitive information can be presented such as airline 
schedules, cultural and other events, etc. Thus, for example a caller asking for flight times 
5 to New York from a 650-493 -#### telephone number might be prompted to select one of 
the three local airports: San Francisco International, San Jose Intemational, and Oakland 
International, and then the flight times to New York after the current time in the Pacific 
time zone would be presented. 

Some additional examples include customizing the presentation of business reports 
10 based on whether or not the market is open; modifying the greeting prompt based on the 
time of day; and providing traffic information automatically during commute hours, but 
not at other times. 

L Targeted Advertising 

Embodiments of the invention support the presentation of targeted advertising, or 
15 other content, to callers of the voice portal 1 10 as shown at step 508. The two primary 

types of targeted advertising supported by embodiments of the invention will be described. 
The different types of targeted advertising can be combined as well. 

1. Based Solely on Telephone Identifying Information 

Telephone identifying information can be used to reference demographic 
20 information about callers from a particular area. For example, if the telephone identifying 
information includes the calling number 650-493-####, corresponding to Palo Alto, 
California, general demographic information about callers from that particular region can 
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be used to target the advertising, or othier content. Further, if a reverse lookup database is 
used, the phone number can, in some instances, locate specific demographic information 
for a given household, or caller. 

This personalization allov^s the targeting of advertising to qualified callers by the 
5 voice portal 112. For exmnple, an advertiser of expensive luxury vehicles might request 
that its callers be qualified based on their income, or a particular psychographic attribute, 
e.g. fuji-loving. In that case, the demographic profile corresponding to the telephone 
identifying information can be used to qualify the caller. Thus, callers from the relatively 
affluent city of Palo Alto, California might receive the advertising. Similarly, if a 
1 0 particular household meets the requirements based on a reverse lookup, those households 
can receive the advertising as well. 

2. Based on Profile 

Another soiirce of information about the caller is the profile used by the shared 
database 112. This profile may indicate interests based on the explicit and implicit 

15 preferences, e.g. likes sports, and can be used in combination with the telephone 
identifying information to more closely tailor ads to the caller. 

For example, if the caller has added movie and entertainment information to 
her/his profile, either explicitly or implicitly, advertising related to movies and 
entertainment could be favored over other qualified advertising based on the caller's 

20 profile. Other examples include providing brokerage, and other financial services, 

advertisements to callers who frequently check stock quotes and/or have a customized 
stock list. 
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J. Adaptive Voice Character 

Turning to step 508, the customization of content through adaptive voice character 
will now be described. The telephone identifying information includes information about 
the caller's locale independent of any user provided registration information. The locales 
5 may be associated with one or more standard voice character settings, e.g. for dialect, and 
also idiomatic speech. Thus, callers from California may receive different prompts and a 
different dialect from the voice portal then callers from Florida. 

Similarly, the telephone identifying information may include information about the 
type of phone, e.g. pay phone, hospital phone, etc., that can be used to adjust the voice 
10 character, e.g. louder and slower speech. 

Additionally, the caller's speaking voice may be used to refine the voice character 
of the system. Thus, callers with speech pattems from a particular region of the country 
may find that after several verbal interactions with the voice portal, the content being 
presented at step 512 is being spoken using a voice character more suited to their own 
15 speech pattems. Similarly, in response to callers who request that information be repeated 
several times, the voice character for those callers may be slowed and played back louder. 
Additional examples include allowing users to select different voice actors, different 
background music and/or sound effects, control the verbosity of prompts, etc. 

K. Purchase Recommendations 

20 Turning to step 510, the customization of content through purchase suggestions. 

Based on the caller's profile ~ as retrieved through the telephone identifying information ~ 
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and/or demographic information from other sources, e.g. locale based and/or reverse 
lookup, the system can make purchasing suggestions. 

The suggestions could be based on the caller's locale and what others in that locale 
have purchased. In other embodiments, the suggestions may be based on the profile of the 
5 user relative to other user's purchases. In some embodiments, approaches such as 
collaborative filtering are used to generate recommendations. 

Examples of recommendations may include particular goods and services, e.g. 
flowers for Mom a few days before Mother's Day. Further, the exact suggestion may vary 
based on the caller's past habits, e.g. in the past you bought chocolates so this year 
10 chocolates might be suggested again. Alternatively, if many people from your locale are 
buying a particular book that might be suggested as well. The particular purchase 
recommendation may relate to goods and services offered independently, by, and/or in 
affiliation wdth the operator of the voice portal 110. 

L. Voice Login 

15 As discussed above, the system may support the use of one or more passwords, 

either spoken or touch-tone for login and authentication purposes. The passwords provide 
for protection against modifications to a user's profile without authentication. 
Additionally, certain specific actions, e.g. making a purchase, listening to certain types of 
personalized content, etc., may require authentication. 

20 Typically, the authentication system will support either a voice or a touch-tone 

password for users of the voice portal 110. This flexibility addresses situations where the 
voice password is not working due to line conditions and/or conditions of the calling 
telephone. Products such as Nuance Verifier™ and/or other voice login and verification 
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products may be used to provide the voice login feature. In some embodiments, both types 
of authentication may be required. 

Once logged in, or authenticated, embodiments of the invention may minimize the 
need for the user to re-authenticate herself/himself, as described above. Additionally, the 
5 password, either voice and/or touch-tone, used for authentication for telephone sessions 
may be the same or different as any passwords used for authentication for web access to 
the profile customization options described in conjunction with Figure 4. 

M. Automatic Profile Initialization 

As discussed in conjunction with Figure 3, it may be desirable to initialize the 
10 profile using the telephone identifying information. The telephone identifying information 
can be used to select an appropriate demographic profile and list of topics based on the 
calling locale. In other embodiments, a reverse lookup of the calling number provided with 
the telephone identifying information is used to obtain a specific demographic profile for a 
caller and/or her/his household. 
1 5 Then the demographic information derived from the locale and/or the reverse 

lookup are used to set initial profile values. For example, the user's income might be 
estimated based on the average income for the calling locale, e.g. Palo Alto, California, or 
fi-om demographic information fi-om the reverse lookup. Similarly, the caller's initial topics 
might be selected based on commonly selected topics for her/his locale and/or the 
20 preferences available based on the demographic information retrieved by the reverse 
lookup. 

These initial values may be revised based on a caller's later actions. For example, if 
the initial estimate of a caller's age is too high, later actions may cause that information to 
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be revised. Similarly, callers may be permitted to explicitly provide certain types of 
demographic information as needed. For example, the user might provide her/his birth 
date to a horoscope feature provided by the voice portal 1 10, in that instance, the birth date 
might be incorporated into the profile. 

5 N. Streaming Content Delivery 

The preceding discussion has focused on personalization of content for users 
accessing the voice portal 110. This section describes special treatment of streaming 
content v^thin various embodiments of the invention. Fig. 6 illustrates an example use of 
the system of Fig. 1. to deliver streaming content to a telephone user. Note that some of 
10 the blocks in Fig. 6 can be executed in different order and/or performed in parallel. 

At block 610, the execution engine 202 receives a request to access an Internet site. 
This request could come from a number of different sources (e.g., a voice command from 
the user, a selection by the user of a menu of possible sites, a command from another part 
of the voice portal 110 because of some personalization choices made by the user). What 
15 is important is that some Internet related request is received. In some embodiments, the 
request may correspond to a URI (Uniform Resource Identifier), Note, if the URI 
corresponds to non-streaming data, then the voice portal 110 acts as described above. 

At block 620, information about the streaming content is determined. The type and 
source of streaming content can have different effects on the streaming engine 224. For 
20 example, if the content stream complies with Real Network's standard, then the streaming 
engine 224 may use the RealPlayer client functional code. The streaming engine 224 could 
decide to cache a large part of the streaming content if the streaming content server is not 
local. Additionally, the streaming engine 224 may notify the server to send only the audio 
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streaming content. What is important is that the streaming engine 224 may make decisions 
about how to access and process the content stream based upon the type and/or location of 
the Intemet site being accessed. 

At block 630, the voice portal 110 begins to receive the streaming content. This 
5 can include receive packets of streaming content and converting the streaming content into 
digital audio data. Alternatively, no conversion is needed where the streaming content is 
sent in a format compatible with internal representation of the audio data within the voice 
portal 1 10. Note, only a portion of the entire stream is processed at any one time. The 
stream is received in parts and, as such, is processed in parts. 

10 At block 640, the execution engine may cause the audio data may be mixed with 

additional audio data. This may be done when user menu prompts/notices from the voice 
portal 1 10 are supplied overtop of the audio from the streaming audio signal. This mixing 
may be done in the telephone gateway 107 and/or in the computers running the voice 
portal 110. Examples of the types of audio signals mixed can include streaming content 

15 from the local streaming content server 160, streaming content from the streaming content 
server 150, audio from the text to speech engine 214, and/or audio content from the audio 
repository 212. This allows prompts, system notifications, menu options, and other user 
interface features to be combined with streaming content delivery. The mixing may be 
accomplished by reducing the power level of the streaming content and adding the second 

20 signal. Mixing may also be achieve by inserting an audio signal into the stream. This 

allows for the insertion of advertisements, systems prompts, etc. Note that, like the audio 
signal from the streaming content, the audio retrieved from the local streaming content 
server 160, the text to speech engine 214, and/or the audio repository 212, may need to be 
converted to a form compatible with the representation of audio within the voice portal 
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110. For example, the audio repository 212 may store audio signals in many different 
forms (e.g., .WAV form, MPS, MIDI, etc.), these forms may need to be converted into the 
internal representation of audio signals within the voice portal 1 10. This conversion may 
also include resolving a location of stored audio information. For example, if the audio 
5 repository 212 refers to audio data stored at another location (e.g. on the Internet) , the 
voice portal 110 may need to resolve the address, download the audio signal, and then 
convert the signal to the appropriate intemal representation. Note, the voice portal 110 
may use multiple intemal representations of audio data. 

At block 650, the mixed audio signal is sent from the telephone gateway 107, 
10 through the PSTN, to the user's telephone. 

At block 660, a test is made to determine whether the entire stream has been 
received. If not, block 630 through 660 are repeated. 

Thus, the user receives streaming content from the Intemet on their telephone. The 
following describes important aspects of the invention that enhance the user's experience 
1 5 with the voice portal 110. 

1. User Controls 

Through touch tone and/or voice commands, the user can control the delivering of 
the streaming content. Various embodiments of the invention allow for variations and 
combinations of the following commands: 
20 • Volume control - increase/decrease the volume of the audio 

• Start/stop/pause/skip/fast forward/rewind - allows the user to control the stream. Some 
of these commands use caching of the content by the voice portal 110. Where the 
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streaming content server supports these commands, the streaming engine 224 may pass 
these commands to the server. 

• Stream location - indicates how far into the stream the user is and/or how long the 
stream is and/or how much is left in the stream. 

• Switch streams - allows the user to switch to another content stream 

• Restore - allows the user to restore a previous stream to the user's last location. 

• Rate - indicates the rate at which the content stream is being received by the voice 
portal 110. 

• Jump - allows the user to jump in the stream to subject matter of interest. To use this 
command, the voice portal 110 performs voice recognition on the content stream (this 
may involve caching portions of the content stream) and searches for the 
corresponding phrase, subject matter, word, etc. 

• Extra - this command tells the user about the metadata for a stream. The data may 
include the length, title, source, and/or server type. 

2. Additional Features 

This section describes additional features of various embodiments of the invention. 

The voice portal 110 can support channels. The channels correspond to specific 
content types such as business, sports, etc. The channels may include streaming data. A 
user could select a channel and have data from various sources combined and delivered to 
him/her. 

History capabilities may be included. This allows the user to select from previously 
accessed Internet sites to replay streaming content or new content from the same site. 
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To support the business, the voice portal 110 may insert advertisements in the 
streaming content. This may be done by adding advertisements before, during, or after, the 
streaming content is delivered to the user. The ads may be determined from the user's 
preferences. 

5 Some embodiments of the invention may use text feeds as part of their streaming 

content. For example, some embodiments of the invention may use the closed caption 
translation of a portion of a audio broadcast to provide streaming translation facilities to a 
user. 

Some embodiments of the invention allow for the mixing of audio from sources 
10 other than the local streaming content server 160, the text to speech engine 214, or the 
audio repository 212. As noted above, the voice portal 110 may access non-streaming 
audio data from the Intemet to mix with streaming content audio signals. 

O. Conclusion 

Thus, a voice portal for presenting streaming content over a telephone interface has 
1 5 been described. The streaming content may be supplied from the Intemet and may be 

combined with other signals. PersonaUzation of the data may also occur. The personalized 
information provided over the voice portal may come from the World Wide Web 
(WWW), databases, third parties, and/or other sources. Telephone identifying information 
is used by embodiments of the invention to personalize caller interactions with the voice 
20 portal. 

The foregoing description of various embodiments of the invention has been 
presented for purposes of illustration and description. It is not intended to limit the 
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invention to the precise forms disclosed. Many modifications and equivalent arrangements 
will be apparent. 
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CLAIMS 

What is claimed is: 



1 1 . A method of providing streaming content from the Internet to a telephone using a 

2 computer system, the computer system including a telephone interface system coupled in 

3 conmiunications with an Internet access system, the telephone interface system being 

4 coupled in communications with the telephone, the method comprising: 

5 receiving an Internet access request, the Internet access request corresponding to 

6 an Internet site outside of the computer system; 

7 receiving the streaming content from the Intemet site, the streaming content 

8 including an audio portion; and 

9 sending at least the audio portion of the streaming content over the telephone 

10 interface system to send an audio signal, corresponding to the audio portion, to 

1 1 the telephone. 

1 2. The method of claim 1 , wherein the receiving the Intemet access request comprises 

2 receiving a verbal request to access the Intemet site and performing voice recognition on 

3 the verbal request to determine the Intemet access request. 

1 3. The method of claim 1, wherein the receiving the Intemet access request comprises 

2 receiving a series of one or more touch tone signals and decoding the series of one or more 

3 touch tone signals to determine the Intemet access request. 
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1 4. The method of claim 1 , further comprising providing a menu corresponding to 

2 accessible Internet sites, and wherein the receiving the Internet access request corresponds 

3 to receiving a selection from the menu. 

1 5. The method of claim 1, further comprising determining the type of streaming 

2 content and converting the corresponding type of streaming content into an audio portion. 

1 6. The method of claim 1 , wherein the Internet access system includes a web server, 

2 wherein the web server connects to the Internet site to receive the streaming content. 

1 7. The method of claim 1 , wherein the streaming content is received as packet data 

2 from a packet switched network and wherein the telephone interface system communicates 

3 the audio signal to the Public Switched Telephone Network. 

1 8. The method of claim 1 , wherein the Internet access request is translated into a URI. 

1 9. The method of claim 1, wherein the computer system includes personalized 

2 content information and wherein the Internet access request is generated from the 

3 personalized content information. 

1 10. The method of claim 1 , wherein the computer system further comprises a local 

2 streaming content system, the local streaming content system including a second 

3 streaming content, the second streaming content including at least a second audio portion, 

4 and wherein the method fiirther comprises accessing the local streaming content system to 

5 provide a second audio signal corresponding to the second audio portion to the telephone. 
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1 11. The method of claim 1 0, further comprising mixing the audio portion with the 

2 second audio portion and wherein the audio signal corresponds to the mix. 

1 12. The method of claim 10, wherein the computer system further comprises an audio 

2 repository, wherein the method further comprises accessing the audio content from the 

3 repository to provide to the telephone, and mixing the audio portion with the second audio 

4 portion and the audio content. 

1 13. The method of claim 1 , wherein the computer system further comprises an audio 

2 repository and wherein the method further comprises accessing the audio content from the 

3 repository to provide to the telephone. 

1 14. The method of claim 1, further comprising receiving a command, the command 

2 corresponding to a request by the telephone user to adjust the sending of the audio portion, 

3 and adjusting the sending of the audio portion. 
4 

1 15. The method of claim 13, wherein the command corresponds to a pause command, 

2 and wherein the sending the audio portion is paused and wherein a pause command is sent 

3 out to the Internet site. 

1 16. The method of claim 1 , wherein the command corresponds to a pause command, 

2 and wherein the sending the audio portion is paused and wherein at least the audio portion 

3 of the streaming content is cached during the pause. 
4 
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1 17. A computer system to deliver streaming content from the Internet to a telephone, 

2 the computer system comprising: 

3 an Internet interface including at least one program to receive the streaming 

4 content from the Internet and extract a streaming audio signal from the 

5 streaming content; 

6 a telephone interface to send an audio signal to the telephone, the audio signal 

7 corresponding to the streaming audio signal; and 

8 a control subsystem to control the Internet interface and the telephone interface. 

1 18. The computer system of claim 17, fiirther comprising an audio repository storing 

2 audio storing sounds, and the computer system includes a second program to cause at least 

3 some sounds to be mixed with the streaming audio signal. 

1 19. The computer system of claim 1 8, wherein the second computer mixes by adding 

2 the at least some sounds to the streaming audio signal, 

1 20. The computer system of claim 18, wherein the second computer mixes by inserting 

2 the at least some soimds into the streaming audio signal. 
3 

1 21 , The computer system of claim 1 8, wherein at least one sound is an advertisement. 

1 22. The computer system of claim 21, wherein the computer system includes personal 

2 preference information and wherein the advertisement is chosen based at least partially 

3 upon the personal preference information. 
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1 23 . The computer system of claim 1 8, wherein at least one sound is a system prompt. 

1 24. The computer system of claim 17, wherein the telephone interface subsystem 

2 includes a call manager, the call manager supporting multiple simultaneous telephone calls 

3 over the telephone interface, at least one of the simultaneous telephone calls receiving the 

4 streaming audio signal. 



1 25. A computer system delivering streaming content to a telephone, the streaming 

2 content being received from a Intemet streaming content server, the computer system 

3 comprising: 



5 



4 



means for receiving an Intemet access request, the Intemet access request 



5 



corresponding to an Intemet site outside of the computer system; 



6 



means for receiving the streaming content from the Intemet streaming content 



7 



server, the streaming content including an audio portion; and 



8 



means for sending at least the audio portion of the streaming to the telephone. 
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ABSTRACT 

A method and apparatus for providing streaming content over telephones is 
described. The creation of a voice portal is supported by the invention. Embodiments of 
the invention use allows users to place a telephone call to access the voice portal. The user 
can access many different types of content. This content can include text based content 
which is read to the user by a text to speech system (e.g., news reports, stock prices, text 
content of Intemet sites), audio content which can be played to the user (e.g., voicemail 
messages, music), and streaming audio content (e.g., Intemet broadcast radio shows, 
streaming news reports, and streaming live broadcasts). This content can be accessed from 
many different places. For example, the content can be retrieved from a news feed, a local 
streaming content server, an audio repository, and/or an Intemet based streaming content 
server. The streaming content allows the user to access live web broadcasts even though 
the user may not have access to a computer. The system allows audio signals from 
multiple sources to be combined to enhance the user experience. Additionally, the user can 
control many aspects of the delivery of the streaming content. Personalization features of 
the voice portal allow for an improved user experience during streaming content delivery. 
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I believe I am the original, first and sole inventor (if only one name is listed below) or 
an original, first and joint inventor (if plural names are listed below) of the subject matter 
which is claimed and for which a patent is sought on the invention entitled: 

STREAMING CONTENT 
OVER A TELEPHONE INTERFACE 

the specification of which 

X is attached hereto. 



was filed on as Application No. 

and was amended on f . 

(If Applicable) 

I hereby state that I have reviewed and understand the contents of the above-identified 
specification, including the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose information which is material to the examination 
of this application in accordance with Title 37, Code of Federal Regulations, §1.5 6(a) which 
states in relevant part: "Each individual associated with the filing and prosecution of a patent 
application has a duty of candor and good faith in dealing with the Office, which includes a 
duty to disclose to the Office all information known to that individual to be material to 
patentability as defined in this section.... The duty to disclose all hiformation known to be 
material to patentability is deemed to be satisfied if all information knovm to be material to 
patentability of any claim issued in a patent was cited by the Office or submitted to the Office 
in the manner prescribed by §§ 1.97(b)-(d) and 1,98," 

I hereby claim foreign priority benefits xxnder Title 35, United States Code, §119 of 
any foreign application(s) for patent or inventor's certificate as indicated below and have also 
identified below any foreign application for patent or inventor's certificate on this invention 
having a filing date before that of the application on which priority is claimed: 

Prior Foreign Application(s) Priority Claimed 



(Number) (Country) (Day/MonthA^ear Filed) Yes No 



C:\NRPORTBL\PALIB1\DRH\109856S.1 



Attorney Docket No.: 22379-702 

Page 2 



I hereby claim the benefit imder Title 35, United States Code, §120 of any United 
States application(s), and under §1 19(e) of any United States provisional application(s), listed 
below and, insofar as the subject matter of each of the claims of this application is not 
disclosed in the prior United States application in the manner provided by the first paragraph 
of Title 35, United States Code, §112, 1 acknowledge the duty to disclose material 
information as defined in Title 37, Code of Federal Regulation, § 1.56(a) which occurred 
between the filing date of the prior application and the national or PCT international filing 
date of this application: 

Unassigned October 22. 1999 Pending 

(Application Serial No.) (Filing Date) (Patented, Pending, Abandoned) 



(Application Serial No.) (Filing Date) (Patented, Pending, Abandoned) 

I hereby appoint the following attomey(s) and/or agent(s) to prosecute this application 
and transact all business in the Patent and Trademark Office connected therewith, and to file, 
prosecute and to transact all business in connection with international applications directed to 
said invention: 



Address all correspondence to: 

Kent R. Richardson 

Wilson Sonsini Goodrich & Rosati 

650 Page Mill Road 

Palo Alto, CA 94304 

Direct all telephone calls to Kent R. Richardson at (650) 493-9300, 



Paul Davis 
John J. Bruckner 
David J. Weitz 
Kent R, Richardson 
U.P. Peter Eng 
Barbara Courtney 
Richard L. Gregory, Jr. 
Van Mahamedi 
Shantanu Basu 
Shaalu Mehra 
Kenta Suzue 
Shirley Chen 
Joel Harris 



29,294 

35,816 

38,362 

39,443 

39,666 

42,442 

42,607 

42,828 

43,318 

P44,934 

P45,145 

P44,608 

P44,743 
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I hereby declare that all statements made herein of my own knowledge are true and that 
all statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willfal false statements and the like so made 
are punishable by fine or imprisonment, or both, under Title 18, United States Code, §1001 
and that such willful false statements may jeopardize the validity of the application or any 
patent issued thereon. 



Full name of sole or 
first inventor: 

Inventor's signature: 

Date: 

Citizenship: 

Residence: 

Post Office Address: 



Hadi Partovi 



United States 



2280 Green Street #104, San Francisco. California 94123 
Same As Above 



Full name of second joint 
inventor, if any:: 

Inventor's signature: 

Date: 

Citizenship: 

Residence: 

Post Office Address: 



Michael S, McCue 



United States 



400 Surmont Road. Los Gatos. Califomia 



Same As Above 



Full name of third joint 
inventor, if any:: 
Inventor's signature: 
Date: 

Citizenship: 

Residence: 

Post Office Address: 



Angus Macdonald Davis 



United States 



991 The Dalles Avenue. Sunnwale. CaHfomia 94087 
Same As Above 
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Full name of fourth joint 

inventor, if any:: 
Inventor's signature: 
Date: 

Citizenship: 
Residence: 

Post Office Address: 

Full name of fifth joint 

inventor, if any:: 
Inventor's signature: 
Date: 

Citizenship: 

Residence: 

Post Office Address: 



Anthony Joseph Accardi 



United States 

1945 Mount Vernon Court. Apt. 16. 
Mountain View. California 94040 
Same As Above 



Michael MacDougall Plitkins 



United States 



1177 Sandia Avenue. Sunnyvale. California 94089 
Same As Above 
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